The rapid development of drones in recent years has ushered in various applications including object detection, patrol, and aerial photography. However, they are still unable to physically interact with targets to complete more complex operations. A drone equipped with an articulated robot arm may perform such complex interactions, such as grasping and holding objects.
In most environments, aerial manipulator will require independent information obtaining and processing abilities; namely, the development of a visual system that can accurately and consistently identify and locate a target is a major focus of research. Researchers at the Nanjing University of Aeronautics and Astronautics have designed a visual recognition and positioning algorithm based on the YOLOv5 deep learning object detection model and the RGB-D sensor, which is able to detect objects and estimate their location in real-time as the robotic arm operates.
As the team's existing robotic arm is unable to complete a full in-flight grip control experiment, the researchers tested the target positioning estimation algorithm in a controlled indoor environment with a simplified experiment. Reflective markers were placed on the drone’s camera arm and the target object; the NOKOV motion capture system was used to record the positions of the markers.
The NOKOV motion capture system was able to output the locations of both the camera and the target in real time through the world coordinate system. Because the NOKOV motion capture system reaches a sub-millimeter level of accuracy, the data obtained by the system was used to evaluate the accuracy of the drone’s vision system after a form-fitting of cloud data and coordinate conversion. The researchers moved the camera arm in one direction to simulate the movement of the camera during flight, thus testing the overall performance of the positioning algorithm.
Bibliography:
Room820, China Minmetals Tower, Chaoyang Dist., Beijing
info@nokov.cn
+ 86-10-64922321
